Non-uniform blind deblurring of dynamic scenes has always been a difficult problem in the field of image restoration. Aiming at the problem that the current blurred image restoration algorithms cannot solve the problem of diverse fuzzy sources well， an end-to-end motion blurred image restoration algorithm based on multi-scale network was proposed. In the proposed algorithm， the pruned residual blocks were used as the basic units， and the same asymmetric encoder-decoder network was used at each scale. In order to extract the features of the input image better， a residual module with attention mechanism was used in the coding side， and a spatial pyramid pooling layer was added. The recurrent unit between the encoding side and decoding side was able to obtain spatial information of the image， so that the image space continuity was able to be used to restore non-uniform motion blurred image. Test results show that the Peak Signal-to-Noise Ratio （PSNR） and Structural SIMilarity （SSIM） of the proposed algorithm are 33.69 dB and 0.953 7 respectively on GoPro dataset， and the blur image details can be recovered better， and the PSNR and SSIM of the proposed algorithm on Blur dataset are 31.47 dB and 0.904 7 respectively. Experimental results show that compared with scale-recurrent network and deep stacked hierarchical multi-patch network， the proposed algorithm achieves better blurred image restoration.